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Standard Method 

Least- Squares estimator of Am 2 and 0: X^ in 

31,32 

FCj = theoretical rate for experiment or bin j 

Rj 3 ^ = rate measured in experiment or bin j 

j = 1, . . . , A/" exp , where N exp is the number of data points 

V = covariance matrix: experimental and theoretical uncertainties 

Standard Goodness of Fit 

Probability to observe a minimum of X 2 larger than the one actually 
observed, assuming for X^ in a x 2 distribution with A^of = N exp — 
-Wpar degrees of freedom (iV par is the number of fitted parameters). 

Standard Allowed Regions 

The standard 100/3% CL regions in the tan 2 $-Am 2 plane are given 
by the condition 

X 2 < X n 2 lin + AX 2 (/3) 
P = Confidence Level (CL) 

AX 2 (f3) = value of % 2 such that the cumulative % 2 distribution for 
2 degrees of freedom is equal to /3 

p = 90% (1.64 a) => AX 2 (0.90) = 4.61 
/3 = 99% (2.58 a) AX 2 (0.99) = 9.21 
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Analysis of Rates 



Rates of Homestake jj], GALLEX+SAGE g, gj, Super- 
Kamiokande 2001 g] N exp = 3, N par = 2, N doi = 1. 

Global Analysis 

Rates of Homestake jj], GALLEX+SAGE [0, g, Super- 
Kamiokande 2001 [B]: 3 data points 

+ 

Shape of Super-Kamiokande 2000 [5j Day-Night data: 6 bins 
and 1 normalization factor 

+ 

Shape of Super-Kamiokande 2001 [gj] energy spectrum for E > 
5.5 MeV: 18 bins and 1 normalization factor 

A/exp = 27, jV par = 4, iV dof = 23. 
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NOVELTIES 



Super- Kamiokande rate/BP-SSM has decreased from 0.474 ± 0.020 
i to 0.451 ± 0.008 g. 

Active Rates 

Best Fit in VO region! Due to decrease of SK rate/BP-SSM. 

Active Global 

Best Fit continues to be in LMA region! 

No SMA region at 99% CL! Due to flat spectrum. 

At 99% CL VO region almost vanish! Due to flat energy 

spectrum and Day-Night asymmetry (albeit small). 

Sterile Rates 

Best Fit continues to be in SMA region! 

Sterile Global 

Poor Goodness of Fit =^> sterile disfavored! 
Best Fit in LMA region! Due to flat energy spectrum and 
decrease of Super-Kamiokande rate/BP-SSM (the incompatibility of 
a flat energy spectrum with the Homestake rate is alleviated). 
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Conditions for the validity of the Standard Method 

\l\ The theoretical rates Rj depend linearly on the parameters 
Am 2 and tan 2 # to be determined in the fit. 

2] The errors Rj — Rj are multinormally distributed. 

3] The covariance matrix V does not depend on Am 2 and tan 2 (9. 
In reality these three conditions are not satisfied: 

\T\ The theoretical rates Rj do not depend at all linearly on the 
parameters Am 2 , tan 2 #. This is the reason why there are several 
allowed regions in the tan 2 #-Am 2 plane and these regions do not 
have elliptic form. 

~2\ The errors Rj — R^ xp ^ are not multinormally distributed, be- 
cause although the fluxes SM and the cross sections Cfj are as- 
sumed to be multinormally distributed, their products, that deter- 
mine the theoretical rates through the relations 

i 

are not multinormally distributed. 



3j The covariance matrix V depends on Am and tan 0. 



6 



{l\ is important! 

[2] is irrelevant : the multinomial approximation is very good, as 
shown by the following three figures. 

Dotted lines: probability distribution function of experimental 
rates generated with Monte Carlo. 

Solid lines: probability distribution function of experimental rates 
assuming a multinormal distribution given by the Likelihood function 
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3j is not negligible, as shown by the following comparison of the 



Standard Regions with the regions obtained with the log- Likelihood 
method (see [0]) 

, r , r AX 2 (/3) 
ln<C > ln£ max 
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Monte Carlo Goodness of Fit [j8j] 

► Estimate best-fit values of Am 2 , tan 2 # through X 2 lin . 

► Call the best-fit values Am 2 , tan 2 #. 

► Assume that Am 2 , tan 2 # are reasonable surrogates of the true 
values Am 2 rue , tan 2 # tr ue- 

► Using Am 2 , tan 2 #, generate N s synthetic random data sets with 
the standard gaussian distribution for the experimental and theoret- 
ical uncertainties. 

► Apply the Least-Squares method to each synthetic data set, lead- 
ing to an ensemble of simulated best-fit parameters Am 2 ( s ), tan 2 #( s ) 
with s = 1, . . . jN S} each one with his associated (X 2 lin ) s . 

► Calculate GoF as the fraction of simulated (^ min ) s in the ensemble 
that are larger than the one actually observed, X min . 
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Goodness of Fit 


Standard 


Monte Carlo 


Active 
Rates 


X 2 • ~ 1 2 

An? ~ 9 x 10" 11 eV 2 
tan 2 19 ~ 0.45 


27% 


8% 


Active 
Global 


X 2 - ~ 24 

mm 

Am 2 ~ 5 x 10" 5 eV 2 
tan 2 ^ ~ 0.40 


40% 


37% 


oteriie 
Rates 


X 2 - ~ 97 

Am 2 ~4x 10" 6 eV 2 

tan 2 ^ ~ 2 x 10~ 3 


32% 


21% 


Sterile 
Global 


Y 2 ~ S7 

^min — °' 

An? ~ 3 x 10" 5 eV 2 
tan 2 ^ ~ 0.8 


3% 


2% 



Standard method overestimates GoF! 



Explanation: GoF is the probability to obtain better 
fits than the observed one. 

When there are several local minima of X 2 with rela- 
tively close values of X 2 , there are more possibilities to 
obtain good fits and the true goodness of fit is smaller 
than the standard one (obtained assuming that X 2 is a 
quadratic function of tan 2 Am 2 , with only one min- 
imum 
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Monte Carlo CL of 
Standard Allowed Regions [@] 

Definition: 100/3% CL Allowed Regions belong to a set of allowed 
regions that cover the true value of the parameters with probability 

p. 

i Given the usual "100/3% CL" allowed regions in the tan 2 $- 
Am 2 plane, calculate their Monte Carlo Confidence Level /3mc with 
a method similar to_the onejrsed for the Goodness of Fit. 

► Assume that Am 2 , tan 2 # are reasonable surrogates of the true 
values Am 2 rue , tan 2 # true . 

► Generate a large number of synthetic data sets. 

► Apply the standard procedure to each synthetic data set and 
obtain the corresponding "100/3% CL" Standard Allowed Regions in 
the tan 2 #-Am 2 plane. 

► Count the number of synthetic "100/3% CL" Standard Allowed 
Regions that cover the assumed surrogate Am 2 , tan 2 # of the true 
values. 

► The ratio of this number and the total number of synthetically 
generated data set gives the Confidence Level /3m c of the "100/3% 
CL" Standard Allowed Regions. 
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CL of Standard Allowed Regions 


standard 


Monte Carlo 


Active 
Rates 


90.00/0 (1.64 a) 


84.08/0 (1.41 cr) 


99.00% (2.58 cr) 


98.16/0 (2.36 cr) 


Active 
Global 


90.00/0 (1.64 a) 


82.83/0 (1.37 cr) 


99.00/0 (2.58 cr) 


97.56/0 (2.25 cr) 


Sterile 
Rates 


nn nnOy /i /° a \ 

90.00/0 (1.64 a) 


87.11% (1.52 cr) 


99.00% (2.58 a) 


98.51% (2.43 a) 


Sterile 
Global 


90.00% (1.64 a) 


80.01% (1.28 cr) 


99.00% (2.58 a) 


97.09% (2.18 a) 



The Confidence Level of Standard Allowed Regions is 
smaller than its nominal value. 



Explanation: When there are several local minima 
of X 2 with relatively close values of X 2 , in repeated ex- 
periments the global minimum has significant chances 
to occur far from the true value of tan 2 # and Am 2 =^> 
smaller probability that the allowed regions cover the 
true value of tan 2 $ and Am 2 , with respect to the case 
in which there is only one minimum. 
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Frequentist Allowed Regions 

Frequentist Statistics allows to calculate allowed regions 
with correct coverage using Neyman's method. 
But there is arbitrariness in the choice of 

1) Estimator of the parameters 

2) Method for the construction of acceptance regions 

i In [@] we have calculated "exact" confidence regions using as 
estimate of tan 2 $ and Am 2 their value at X^ in . 

h In [gj it has been argued that X^ in may be an insufficient esti- 
mator, leading to a loss of information. Notice that if this is true, 
the standard % 2 method suffers from the same problem! 

^ In order to prevent any loss of information, it is better to use 
the full data set as estimator of tan 2 1!) and Am 2 , as done in j9j. 

However, there is still the problem of choice of the method for 
the construction of acceptance intervals. 

A In [{| it has been argued that the Unified Approach (UA) jffj 
is more appropriate than the smallest acceptance intervals method, 
also known as "Crow-Gardner" (CG). 
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=£> Unfortunately, it is well known that when the UA differs from 
the smallest acceptance intervals method it gives unreliable confi- 
dence intervals (see jtt], |3^1 ) 

^> Infamous example: 
The KARMEN 1998 limit on — ► v e oscillations obtained with 
the Unified Approach was unreliably much more stringent than the 
sensitivity of the experiment jt3j . 

The KARMEN 1999 limit is less stringent than the 1998 one. More 
data less information! 
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Other infamous examples: 

The 1999 limit on neutrinoless double-beta decay obtained in 
the Heidelberg-Moscow experiment jtSj obtained with the Uni- 
fied Approach was much more stringent than the sensitivity of the 
experiment. That is why now they do not use the Unified Approach 
any more [|t§]! (Got burned!) 

The present upper limit on — > v T neutrino oscillations ob- 
tained in the NOMAD experiment jT7j is stronger than the one ob- 
tained in the CHORUS experiment [|t8|] not because the NOMAD ex- 
periment has a better sensitivity than the CHORUS experiment (see 
discussion in but because the NOMAD collaboration uses the 
Unified Approach, which gives unphysically stringent upper bounds 
when the number of observed events is smaller than the expected 
background. 
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i The UA and similar methods |tt| |2tj, |2tj are appropriate in order 
to get allowed regions even in the presence of an unlikely statistical 
fluctuation of the data, such that the data are very unlikely for any 
value of the parameters. 

However, the physical reliability of such allowed regions is highly 
questionable. 

i If there is no statistical fluctuation of data, the UA and the CG 
methods are equivalent. 

i From the value of the GoF (see Table at pag. [TT]) one can see 
that there is no unlikely statistical fluctuation in solar 
neutrino data in the case of v e — ► v^ T oscillations and in the case 
of the analysis of the rates in terms of v e — > v s oscillations. 

On the other hand, if the solar neutrino problem is due to v e — ► v s 
oscillations, there is an unlikely statistical fluctuation of the shape of 
the energy spectrum and the global analysis of solar v data with the 
CG method is unreliable. 

i Therefore, the CG method (that is computationally much easier 
than the UA method) can be applied to the analisis of solar v data 
in terms of v e — + Vn T oscillations and to the analysis of the rates of 
solar neutrino experiments in terms of v e —> v s oscillations. 
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Bayesian Allowed Regions 



=>■ Bayesian Theory allows to calculate the improvement of 
knowledge as a consequence of experimental measure- 
ments (see [|22|]). 

=>■ This is how our mind works and how science improves. There- 
fore, Bayesian Theory is the natural statistical tool for 
scientists. 

► Bayesian probability density function of tan 2 $ and Am 2 after 
measurement of rates Rj* : 

j9(tan 2 6>, Am 2 |^ exp) ) oc £(i^ exp) | tan 2 6>, Am 2 )j9(tan 2 #, Am 2 ) 

£>(tan 2 #, Am 2 ) = prior probability density function 

i Prior knowledge on tan 2 $ and Am 2 : All values are allowed, but 
we know that solar v data are sensitive to different orders of magni- 
tude of tan 2 $ and Am 2 through different mechanisms. 

Flat prior in the log(tan 2 ^)-log(Am 2 ) plane 

4 



£>(tan 2 #, Am 2 


| jR (exp) ) = 

£(^ exp) |tan 2 ^,Am 2 ) 


J £(jR (exp)| 


tan 2 #, Am 2 ) dlog(tan 2 $) dlog(Am 2 ) 
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